Skip to content

test: clean up tests writing to tracked files, prevent future violations - #14772

Merged
Kha merged 5 commits into
masterfrom
test-scratch-ignores
Aug 19, 2026
Merged

Kha merged 5 commits into
masterfrom
test-scratch-ignores

Conversation

@Kha

@Kha Kha commented Aug 13, 2026

Copy link
Copy Markdown
Member

Fixes even more tests writing to tracked files, which can create a mess with version control. Makes tracked files read-only during CI test runs and checks for new untracked and non-ignored files afterwards to prevent future violations.

Kha and others added 4 commits August 8, 2026 10:54
This PR stops the test suite from leaving scratch files inside the tracked source tree, so that a test run no longer shows up as a dirty working copy.

The `ltarStable` Lake test now copies its package into `work/` (already ignored, and the idiom `srcHash` uses) instead of appending a comment to the tracked `Test/A.lean` and dropping `out*.jsonl`, `bundles*.txt`, `staging/` and `Test/A.lean.bak` next to it. The two `compact_closure` compile tests adopt the `_tmp_` prefix that `tests/compile/.gitignore` documents. The remaining cases are cleanup targets that no ignore rule covered: `manifest/lake-packages`, `toml/toml-test`, `bench/size/install`, `clone/lean-toolchain`, `init/mathlib_standards` and `rebuild/Foo/`. Scratch `produced.*` output under `tests/pkg` is now covered by one rule in `tests/pkg/.gitignore` rather than per-package copies.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This PR stops several Lake tests from editing files inside the checked-in source tree, so that a test run no longer shows up as a dirty working copy.

`copy_to_work` in `tests/lake/tests/common.sh` replaces the copy-into-`work/` preamble seven tests had spelled out, and additionally makes the copies writable, since `cp` propagates the source mode. `shake` and `updateUnknown` move onto it too: `shake` copied `input/*` over its own test directory, which had left `DepMain.lean` tracked because `.gitignore` transposed the name to `MainDep.lean`, and `updateUnknown` rewrote its tracked `lakefile.toml` in place and leaked `lakefile.toml.bak` whenever a run died before the restoring `mv`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This PR makes CI fail when a test writes into the checked-in source tree, so that scratch output has to go to an ignored path instead.

Tracked files are made read-only for the duration of the test run, and `git status` is compared before and after it. The two halves catch different things: the comparison finds byproducts a test leaves behind, while the read-only bit also catches a test that edits a tracked file and restores it afterwards, which the comparison cannot see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Windows job runs the test suite as well, and MSYS2 honors `chmod`; `tests/lake/tests/cache` already relies on that. The tree comparison never needed the exemption at all, it only inherited it because `tree-before` is captured in the `chmod` step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Kha
Kha requested review from Garmelon and tydeu August 13, 2026 04:32
@Kha Kha added the release-ci Enable all CI checks for a PR, like is done for releases label Aug 13, 2026
@Kha
Kha requested a review from kim-em as a code owner August 13, 2026 04:32

@tydeu tydeu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🎉 I'm happy to see that most of the Lake tests were already doing this.

ltarStable was fixed in #14700, so you need to rebase on a commit after that.

@github-actions github-actions Bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Aug 14, 2026
@leanprover-bot

Copy link
Copy Markdown
Collaborator

Reference manual CI status:

  • ❗ Reference manual CI can not be attempted yet, as the nightly-testing-2026-08-13 tag does not exist there yet. We will retry when you push more commits. If you rebase your branch onto nightly-with-manual, reference manual CI should run now. You can force reference manual CI using the force-manual-ci label. (2026-08-14 05:55:46)

@github-actions github-actions Bot added the mathlib4-nightly-available A branch for this PR exists at leanprover-community/mathlib4-nightly-testing:lean-pr-testing-NNNN label Aug 14, 2026
@mathlib-lean-pr-testing mathlib-lean-pr-testing Bot added the builds-mathlib CI has verified that Mathlib builds against this PR label Aug 14, 2026
@mathlib-lean-pr-testing

mathlib-lean-pr-testing Bot commented Aug 14, 2026 •

Copy link
Copy Markdown

Mathlib CI status (docs):

mathlib-nightly-testing Bot pushed a commit to leanprover-community/batteries that referenced this pull request Aug 14, 2026
mathlib-nightly-testing Bot pushed a commit to leanprover-community/mathlib4-nightly-testing that referenced this pull request Aug 14, 2026
@Kha
Kha added this pull request to the merge queue Aug 19, 2026
Merged via the queue into master with commit a25f21e Aug 19, 2026
47 of 49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

builds-mathlib CI has verified that Mathlib builds against this PR mathlib4-nightly-available A branch for this PR exists at leanprover-community/mathlib4-nightly-testing:lean-pr-testing-NNNN release-ci Enable all CI checks for a PR, like is done for releases toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants